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Abstract: We give a detailed analysis of the Gibbs-type entropy notion and its dynamical be- 
havior in case of time-dependent continuous probability distributions of varied origins: related 
to classical and quantum systems. The purpose-dependent usage of conditional KuUback-Leibler 
and Gibbs (Shannon) entropies is explained in case of non-equilibrium Smoluchowski processes. 
A very different temporal behavior of Gibbs and KuUback entropies is confronted. A specific con- 
ceptual niche is addressed, where quantum von Neumann, classical KuUback-Leibler and Gibbs 
entropies can be consistently introduced as information measures for the same physical system. If 
the dynamics of probability densities is driven by the Schrodinger picture wave-packet evolution, 
Gibbs-type and related Fisher information functionals appear to quantify nontrivial power transfer 
processes in the mean. This observation is found to extend to classical dissipative processes and 
supports the view that the Shannon entropy dynamics provides an insight into physically relevant 
non-equilibrium phenomena, which are inaccessible in terms of the KuUback-Leibler entropy and 
tj^ically ignored in the literature. 

Keywords: differential entropy; von Neumann entropy; Shannon entropy; KuUback-Leibler 
entropy; Gibbs entropy; entropy methods; entropy functionals; Fisher information; Ornstein- 
Uhlenbeck process; Schrodinger picture evolution; dynamics of probability densities; invariant 
density. 
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1 Introduction 

Among numerous manifestations of tlie concept of entropy in physics and mathematics, the 
information-theory based entropy methods have been devised to investigate the large time be- 
havior of solutions of various (mostly dissipative) partial differential equations. Shannon, KuU- 
back and von Neumann entropies are typical information theory tools, designed to quantify the 
information content and possibly information loss for various classical and quantum systems in a 
specified state. 

For quantum systems the von Neumann entropy vanishes on pure states, hence one presumes to 
have a complete information about the state of a system. On the other hand, for pure states the 
Gibbs-type (Shannon, e.g. differential) entropy gives access to another information-theory level, 
associated with a probability distribution inferred from a L^(i?") wave packet. 
Although Shannon or KuUback entropies are interpreted as information measures, it is quite 
natural to think of entropy as a measure of uncertainty. In view of the profound role played 
by the Shannon entropy in the formulation of entropic indeterminacy relations |2] , the term 
information, in the present paper is mostly used in the technical sense, meaning the inverse of 
uncertainty. 

In physics, the notion of entropy is typically regarded as a measure of the degree of randomness 
and the tendency (trends) of physical systems to become less and less organized. Throughout the 
paper we shall attribute a more concrete meaning to the term organization, both in the classical 
and quantum contexts. Namely, we shall pay special attention to quantifying, in terms of suitable 
entropy functionals, the degree of the probability distribution (de) localization, and the dynamical 
behavior of this specific - localization uncertainty - feature of a physical system. 

1.1 Notions of entropy 

Notions of entropy, information and uncertainty are intertwined and cannot be sharply differen- 
tiated. While entropy and uncertainty are - to some extent synonymous - measures of ignorance 
(lack of information, uncertainty), the complementary notion of information basically quantifies 
the ability of observers to make reliable predictions about the system, [110 EI: the more aware 
one is about chances of a concrete outcome, the lower is the uncertainty of this outcome. 
Normally, the growth of uncertainty is identified with an increase of the entropy which in turn is 
interpreted as an information loss. Consult e.g. standard formulations of the celebrated Boltzmann 
H-theorem. 

Following Ref. [Sj let us recall that entropy - be it thermodynamical (Gibbs-Boltzmann), dynam- 
ical, von Neumann, Wehrl, Shannon, Renyi, Tsallis or any other conceivable candidate - has an 
exceptional status among physical quantities. As a derived quantity it does not show up in any 
fundamental equation of motion. Generically, there is no a priori preferred notion of entropy 
(perhaps, except for the thermodynamical Clausius case) in physical applications and its specific 
choice appears to be purpose-dependent. 

As an obvious remnant of the standard thermodynamical reasoning, one expects entropy to be 
a state function of the system (thermodynamical notions of equilibrium or near-equilibrium are 
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implicit). This state connotation is a source of ambiguities, since inequivalent notions of the 
system state are used in the description of physical systems, be them classical, thermodynamical 
and quantum. Not to mention rather specialized meaning of state employed in the standard 
information theory, jHIHllZI- 

A primitive information-theory system is simply a bit whose two admissible states are binary digits 
1 and 0. Its quantum equivalent is a qubit whose admissible states are vectors in a two-dimensional 
Hilbert space, hence an infinity of pure states of a two-level quantum system. 
The information theory framework, if extended to more complicated systems, employs a plethora 
of notions of state 0J Ej . As very special cases we may mention a phase-space point as the determi- 
native of the state of a classical dynamical system, or the macroscopic notion of a thermodynamical 
state in its classical and quantum versions, [H]. 

A symbolic mathematical representation of quantum states in terms of wave vectors and/or density 
operators is expected to provide an experimentally verifiable "information" about the system. 
To obtain a catalogue of the corresponding statistical predictions, an a priori choice of suitable 
observables (and thus measurement procedures) is necessary. Then, a casual interpretation of 
entropy as a measure of one's uncertainty about measurable properties of a system in a prescribed 
quantum state may acquire an unambiguous meaning. 

When adopting the state notion to the Hilbert space language of quantum theory, we realize that 
normalized wave functions and density operators, which are traditionally supposed to determine 
the quantum state, allow to extend the notion of entropy to certain functionals of the state of the 
quantum system. The von Neumann entropy 



of a quantum state (e.g. the density operator p), though often infinite, is typically related to the 
degree of departure from purity (e.g. the "mixedness" level) of the state and is particularly useful 
while quantifying measurements performed upon finite quantum systems. 

For a given density operator p, von Neumann entropy is commonly accepted as a reliable measure 
of the information content (about the departure from purity), to be experimentally extracted 
from of a quantum system in a given state. Only under very specific circumstances, like e.g. in 
an optimal "quantum experiment" fUEI which refers to the diagonal density operator (with pi, 
1 < i < N being its eigenvalues), the information gain can be described in terms of both von 
Neumann's and the standard Shannon measure of information: 



Since von Neumann entropy is invariant under unitary transformations, the result exhibits an 
invariance under the change of the Hilbert space basis and the conservation in time for a closed 
system (when there is no information/energy exchange with the environment). Thus, Schrodinger 
dynamics has no impact on the von Neumann encoding of information, see e.g. also for a 

related discussion. 



5(/5) 
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Pure states have vanishing von Neumann entropy {S{p) = "for the pure states and only for 
them", jni) and are normally considered as irrelevant from the quantum information theory per- 
spective, since "one has complete information" about such states. One may even say that a 
pure state is an unjustified over-idealization, since otherwise it would constitute e.g. a completely 
measured state of a system in an infinite Hilbert space, jT3]. A colloquial interpretation of this 
situation is: since the wave function provides a complete description of a quantum system, surely 
we have no uncertainty about this quantum system, pT)] . 

Note that as a side comment we find in Ref. jT3] a minor excuse: this idealization, often employed 
for position-momentum degrees of freedom, is usually an adequate approximation, to be read as 
an answer to an objection of Ref. ^3: "although continuous observables such as the position 
are familiar enough, they are really unphysical idealizations", c.f. in this connection ^Hj for an 
alternative view. 

On the other hand, the classic Shannon entropy is known to be a natural measure of the amount 
of uncertainty related to measurements for pairs of observables, discrete and continuous on an 
equal footing, when a quantum system actually is in a pure state. Hence, a properly posed 
question reveals obvious uncertainties where at the first glance we have no uncertainty. The 
related entropic uncertainty relations for finite and infinite quantum systems have received due 
attention in the literature, in addition to direct investigations of the configuration space entropic 
uncertainty /information measure of L^(i?"') wave packets, JHl, ^H] - |42j . 

The commonly used in the literature notions of Shannon and von Neumann entropy, although 
coinciding in some cases, definitely refer to different categories of predictions and information 
measures for physical systems. In contrast to the exclusively quantum concept of von Neumann 
entropy, the Shannon entropy - quite apart from its purely classical provenance - appears to 
capture a number of properties of quantum systems which cannot be detected nor described by 
means of the von Neumann entropy. 

Obviously, there is no use of Shannon entropy if one is interested in verifying for mixed quantum 
states, how much actually a given state is mixed. On the other hand, von Neumann entropy 
appears to be useless in the analysis of L'^{R) wave packets and their dynamical manifestations 
(time-dependent analysis) which are currently in the reach of experimental techniques, |^ 132] • 
It is enough to invoke pure quantum states in L'^{R^) and standard position-momentum observ- 
ables which, quite apart from a hasty criticism still stand for a valid canonical quantization 
cornerstone of quantum theory, [TH] . 

Those somewhat underestimated facts seem to underhe statements about an inadequacy of Shan- 
non entropy in the quantum context, [TT], while an equally valid statement is that the von Neu- 
mann entropy happens to be inadequate. The solution of the dilemma lies in specifying the 
purpose, see also [12]. 

1.2 Differential entropy 

We are primarily interested in the information content of pure quantum states in and 
thus pursue the following (albeit scope-limited, c.f. [HI IHj for experimental justifications) view: 
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an isolated system is represented in quantum mechanics by a state vector that conveys statistical 
predictions for possible measurement outcomes. 

Consequently, it is the state vector which we regard as an information (alternatively, predictions 
and uncertainty) resource and therefore questions like, [IH]: how much information in the state 
vector or information about what, may be considered meaningful. Let us emphasize that we do 
not attempt to define an information content of a physical system as a whole, but rather we wish 
to set appropriate measures of uncertainty and information for concrete pure states of a quantum 
system. 

A particular quantum state should not be misinterpreted to provide a complete description of the 
corresponding physical system itself, jlHj. In fact, when we declare any Schrodinger's ip as the 
state of a quantum system, we effectively make a statement about the probabilities of obtaining 
certain results upon measurement of suitable observables, hence we refer to a definite experimental 
setup. Therefore, to change or influence this state is not quite the same as changing or influencing 
the system. 

Our, still vague notion of information, does not refer to qubits since we shall basically operate 
in an infinite dimensional Hilbert space. This does not prohibit a consistent use of information- 
theory concepts, since an analytic information content of a quantum state vector, in the least 
reduced to a properly handled plane wave, is not merely an abstraction and can be dug out in 
realistic experiments, including those specific to time- dependent quantum mechanics, |13l Elj . On 
the way one may verify a compliance with quantum theory of a number of well defined properties 
of the quantum system for which: the only features known before an experiment is performed are 
probabilities of various events to occur, [TT] . 

In the case of a quantum mechanical position probability density, its analytic form is assumed 
to arise in conjunction with solutions of the Schrodinger equation. Then, we need to generalize 
the original Shannon's entropy for a discrete set of probabilities to the entropy of a continuous 
distribution with the density distribution function |^ , which is also named the differential entropy. 



Most of our further discussion will be set in a specific context of quantum position-momentum 
information/uncertainty measures, where the classical form of Shannon differential entropy |3] 
has been used for years in the formulation of entropic versions of Heisenberg-type indeterminacy 
relations, jlHl EDI EB ES] • 

The entropic form of indeterminacy relations, enters the stage through the Fourier analysis of 
wave packets, in conjunction with the Born statistical interpretation, hence with ■^/'-induced 
probability measures in position and momentum space, JOlEni- The experimental connotations 
pertaining to the notion of uncertainty or indeterminacy are rather obvious, although they do not 
quite fit to the current quantum information idea of a "useful" quantum measurement, [TT|j . 
Given the probabihty density p{x) on i?", we define the differential entropy [HllSlllHl), as follows: 



One may consider a subset F C i?" to be a support of p instead of R; this is guaranteed by the 



0ini. 




(3) 
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convention that the integrand in Eq. vanishes if p does. Note a minor but crucial notational 
difference between p and p. 

Let us stress that, modulo minor exceptions, throughout the paper we carefully avoid dimensional 
quantities (all relevant dimensional constants like the Planck h are scaled away), since otherwise 
the above differential entropy definition may be dimensionally defective and have no physical 
meaning. An extensive list od differential entropy values for various probability densities can be 
found in Re. 

Since our paper is supposed to be concerned with physical applications, in Section II we shall 
analyze the issue of how the differential entropy definition depends on the units used. The related 
difficulty, often overlooked in the literature, refers to literally taking the logarithm of a dimensional 
argument, see e.g. im]. 

In the quantum mechanical context, we shall invoke either position S{p) or momentum S{p) 
information entropies, with no recourse to the classical entropy given in terms of classical phase- 
space distributions f{q,p) or (Wehrl entropy) their Wigner/Husimi analogues, [SI EH]- The notion 
of entropic uncertainty relations, [3 1211 122 explicitly relies on the differential entropy input. 
Namely, an arithmetic sum of (presumed to be finite) momentum and position information en- 
tropies for any normalized L^(i?") wave packet ipix), is bounded from below: 

S{p)+S{p)>n{l + \n7i) (4) 

where n stands for the configuration space (respectively momentum space) dimension, j2Tj. This 
feature is worth emphasizing, since neither S{p) nor S{p) on their own are bounded from below 
or from above. Nonetheless, both take finite values in physically relevant situations and their sum 
is always positive. 

1.3 Temporal behavior-preliminaries 

Since a normalized wave function represents a pure state of a quantum system whose dynamics 
is governed by the Schrodinger equation, only for stationary states the differential entropy S{p) is 
for sure a conserved quantity. In general, the Schrodinger picture evolution of ip{x,t) and so this 
of |'?/'(x, t)p = p{x,t) may give rise to a nontrivial dynamics of the information entropy associated 
with the wave packet ilj{x,t). 

Let us point out that most of the "entropic" research pertains to time- independent situations, like 
in case of stationary solutions of the Schrodinger equation. Notable exceptions are Refs. j23 IS31 
El] ■ On general non-quantum grounds an information (differential entropy) dynamics is addressed 
in Refs. 0117] and |Si-|ni, see also [HIl EHl EH EH EO] • 

The differential entropy, by a number of reasons IHE], is said not to quantify the absolute "amount 
of information carried by the state of the system" (Shannon's uncertainty), unless carefully inter- 
preted. Up to measure preserving coordinate transformations the latter objection remains invalid 
and this feature gave some impetus to numerically assisted comparative studies of the Shannon 
information content of different pure states of a given quantum system. 

Results are ranging from simple atoms to molecules, nuclei, aggregates of particles, many-body 
Bose and Fermi systems, and Bose- Einstein condensates, see e.g. Refs. [23 - |12]- In these cases. 
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Shannon's differential entropy appears to be a fully adequate measure for the localization degree 
(which in turn is interpreted as both the uncertainty measure and the information content) of the 
involved wave packets. 

A difference of two information entropies (evaluated with respect to the same coordinate system) 
S{p) — S{p') is known to quantify an absolute change in the information content when passing 
from one state of a given system to another. All potential problems with dimensional units would 
disappear in this case, jH IH] . Alternatively, to this end one may invoke the familiar notion of the 
relative Kullback entropy — Jp p (In p — In p') dx, [U E], provided p' is strictly positive. 
Cogent recommendations towards the use of the Shannon information measure, plainly against 
the Kullback option, can be found in Ref. [72]. We shall come to this point later. For arguments 
just to the opposite see e.g. |73j and also [021 EBj- 

In the present paper, we predominantly invoke the differential entropy. In Section IV we shall 
describe a number of limitations upon the use of the Kullback entropy. If both entropies can 
be safely employed for the same physical model (like e.g. for the diffusion type dynamics with 
asymptotic invariant densities), we establish direct links between the Shannon and Kullback en- 
tropy dynamics. 

In the context of the induced (by time development of probability densities) "information dy- 
namics" S S(t), ^ ESI, it is the difference S{t) — S{t') between the (presumed to be finite) 
information entropy values for the time-dependent state of the same physical system, considered 
at times t' < t, which properly captures the net uncertainty /information change in the respective 
time interval [t',t]. Let us mention that the very same strategy underlies the Kolmogorov-Sinai 
entropy notion for classical dynamical systems, [Hni ISH EHj • 

In particular, the rate in time of information entropy ^ is a well defined quantity characterizing the 
temporal changes (none, gain or loss) in the information content of a given L^(i?") normalized wave 
packet ^J^x^t) (strictly speaking, of the related probability density). We indicate that, at variance 
with standard thermodynamical intuitions, quantum mechanical information (differential) entropy 
needs not to be a monotonic function of time. In the course of its evolution it may oscillate, increase 
or decrease with the flow of time, instead of merely increasing with time or staying constant, as 
customarily expected. That, regardless from the intrinsic time reversal property of the quantum 
dynamics. 

To conform with the information theory lore, we need to address an information entropy balance 
in the course of time, since for an isolated quantum system there is no analog of thermal reservoir, 
capable of decreasing (removal) or increasing (putting into) an information entropy of the partic- 
ular state in which the system actually is. Since there in no quantifiable energy exchange with the 
environment, the actual purpose and "fate" of the differential entropy need to be investigated. 
The entropic method we follow in the present paper extends to any formalism operating with 
general time-dependent spatial probability densities, |17l EHJ EO] , even if set out of the explicit 
thermodynamic context (e.g. the phase space formulation of statistical mechanics). 
Information entropy and its intrinsic dynamics, like e.g. the information flow and information 
entropy production rate, quantify properties of general reversible and/or irreversible dynamical 
systems. Normally, the microscopic dynamics of such systems is expected to follow well defined 
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trajectories (deterministic paths of a dynamical system or sample paths of a stochastic process) and 
those may be thought to induce a corresponding dynamics for statistical ensembles of trajectories. 
It is seldom possible to have a sharp wisdom of the initial data Xq E X for the trajectory dynamics 
taking place in a phase space X of the system. This imprecision extends to the terminal data 
{xq — > Xt after time t > 0) as well. 

Therefore, even if one knows exact dynamical rules governing the behavior of individual trajectories 
in time, it is basically impossible to tell more about the system then: if its initial state can be found 
in a subset A G X with a probability prob{xQ G A), then after time t one can identify the terminal 
state of the system X( G X in a subset B G X with a probability prob{xt G B). An evolution of 
derived probability densities eventually may be obtained as a solution of an appropriate partial 
differential transport equation, jUl EHl EHl lEHI 

In the present paper we take a more general view and we bypass a concept of the underly- 
ing trajectory dynamics by emphasizing the role of transport equations and their density so- 
lutions. Under such premises, we can safely address the dynamics of uncertainty /information 
generated by the Schrodinger picture quantum evolution of wave packets in closed (no system - 
reservoir /environment coupling) quantum mechanical systems. 

Remark 1: Keeping in touch with quantum mechanical tradition, let us recall that at least two 
different "trajectory pictures " can be related to the very same mathematical model based on 
the Schrodinger wave packet dynamics: deterministic Bohmian paths [IH] and random paths of 
(basically singular) diffusion-type processes, jSOl EH E2] ■ Additionally, under suitable restrictions 
(free motion, harmonic attraction) classical deterministic phase-space paths are supported by the 
associated with V^(x, t) positive Wigner distribution function and its spatial marginal distribution. 
However, none of the above derived trajectory "pictures" deserves the status of an underlying 
physical "reality" for quantum phenomena although each of them may serve as an adequate 
pictorial description of the wave-packet dynamics. 

Remark 2: In view of Born's statistical interpretation postulate, the Schrodinger picture dynam- 
ics sets a well defined transport problem for a probability density p{x,t) = |?/'(a;,t)p. Therefore, 
one is tempted to resolve such dynamics in terms of (Markovian) diffusion-type processes and 
their sample paths, see e.g. [201 El E2| and [SSIISI]- A direct interpretation in terms of random 
"trajectories" of a Markovian diffusion-type process is here in principle possible under a number 
of mathematical restrictions, but is non-unique and not necessarily global in time. The nontrivial 
boundary data, like the presence of wave function nodes, create additional problems although the 
nodes are known to be never reached by the pertinent processes. The main source of difficulty lies 
in guaranteing the existence of a process per se i.e. of the well defined transition probability den- 
sity function solving a suitable parabolic partial differential equation (Fokker- Planck or Kramers). 
By adopting milder conditions upon the drift fields (instead of too restrictive growth restrictions, 
one may simply admit smooth functions) it is possible to construct well defined, albeit non-unique, 
diffusion-type processes. They are consistent with the time development of a given probability 
density, see Chap. 3 of Ref. [HHI and (S^J. 
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1.4 Outline of the paper 

The paper is structured as follows. We begin by recalling the standard lore of the Shannon infor- 
mation theory to attribute an unambiguous meaning to two principal notions, this of information 
and that of uncertainty. To this end various notions of state of a model system are invoked and 
suitable information measures are discussed. 

Next we turn to the coarse-graining issue and set a connection between the Shannon entropy 
of a discrete probability measure and the differential entropy of a related (through a suitable 
limiting procedure) continuous probability density. On the way, we analyze the dimensional units 
impact on the entropy definition. We discuss various entropic inequalities for both differential and 
coarse-grained entropies of quantum mechanical densities. 

In Section III, the localization degree of probability densities is analyzed by means of so-called 
entropy powers and of the Fisher information measure. We infer two chain inequalities, Eqs. (j3T|) 
and which imply that typically the differential entropy is a well behaved quantity, bounded 
both from below and above. The formalism is general enough to include quantum mechanical 
densities as merely the special case. 

In Section IV we set a conceptual framework for time-dependent problems. Since classical dynam- 
ical, stochastic and quantum systems (in their pure states) in general give rise to time- dependent 
probability densities and information entropies, we resolve the exemplary density dynamics in 
terms of Smoluchowski diffusion processes, albeit with no explicit random path (e.g. random 
variable) input. 

The entropy and Fisher information evolution equations are established. Close links of the dif- 
ferential and conditional Kullback entropies are established for Smoluchowski diffusion processes, 
when asymptotic invariant densities enter the scene. We discuss a compliance of the induced 
continual power release in the course of the diffusion process with the mean energy conservation 
law, Eqs ^TMf and (fTTl^ . 

In section V we analyze differential entropy dynamics and time evolution of the Fisher localization 
measure in quantum theory and next exemplify the general formalism for simple analytically 
tractable cases. The emergent continual power transfer effect has been analyzed in connection 
with the finite energy constraint for the mean energy of quantum motion, Eqs. ()114|) and ()117|) . 
Although uncertainty dynamics scenarios of sections IV and V are fundamentally different, nonethe- 
less the respective methodologies appear to have an overlap, when restricted to steady states which 
support invariant densities for (reversible) stationary diffusion-type processes. 

2 Differential entropy: uncertainty versus information 

2.1 Prerequisites 

The original definition of Shannon entropy conveys a dual meaning of both uncertainty and in- 
formation measure. It is useful to interpret those features in a complementary (albeit colloquial) 
way: the less is the uncertainty of the system or its state, the larger (and more valuable) is the 
information we acquire as a result of the measurement (observation) upon the system, and in 
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reverse. 

We know that a result of an observation of any random phenomenon cannot be predicted a priori 
(i.e. before an observation), hence it is natural to quantify an uncertainty of this phenomenon. 
Let us consider /i = (/xi, ...,/i7v) as a probability measure on distinct (discrete) events Aj, 1 < 
j < N pertaining to a model system. Assume that ^^i/ij = 1 and /ij = prob{Aj) stands for a 
probability for an event Aj to occur in the game of chance with N possible outcomes. 
Let us call — log fij an uncertainty function of the event Aj. Interestingly, we can coin here the 
name of the ("missing") information function, if we wish to interpret what can be learned via 
direct observation of the event Aj: the less probable is that event, the more valuable (larger) is 
the information we would retrieve through its registration . 
Then, the expression 

N 

'5(/i) = -^/ijlog /ij (5) 

i=i 

stands for the measure of the mean uncertainty of the possible outcome of the game, 0, and 
at the same time quantifies the mean information which is accessible from an experiment (i.e. 
actually playing the game). The base of the logarithm for a while is taken equal 2, but we recall 
that log 6 ■ In 2 = In 6 and ln2 ~ 0.69555 with the base e ~ 2.71828 . 

Thus, if we identify event values Ai,...,An with labels for particular discrete "states" of the 
system, we may interpret Eq. (0) as a measure of uncertainty of the "state" of the system, before 
this particular "state" it is chosen out of the set of all admissible ones. This well conforms with the 
standard meaning attributed to the Shannon entropy: it is a measure of the degree of ignorance 
concerning which possibility (event Aj) may hold true in the set {Ai,A2, ...,A]\f} with a given a 
priori probability distribution {fii, ...,fiN}. 
Notice that: 

< 5(/i) < log N (6) 

ranges from certainty (one entry whose probability equals 1 and thus no information is missing) 
to maximum uncertainty when a uniform distribution Hj = 1/N for all 1 < j < occurs. In the 
latter situation, all events (or measurement outcomes) are equiprobable and log A^ sets maximum 
for a measure of the " missing information" . 

By looking at all intermediate levels of randomness allowed by the inequalities Eq. ^ we realize 
that the lower is the Shannon entropy the less information about "states" of the system we are 
missing, i.e. we have more information about the system. 

If the Shannon entropy increases, we actually loose an information available about the system. 
Consequently, the difference between two uncertainty measures can be interpreted as an informa- 
tion gain or loss. 

2.2 Events, states, microstates and macrostates 
The Boltzmann formula 

S = kelnW = -kelnP (7) 
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sets a link of entropy of the (thermodynamical) system with the probabihty P = 1/W that 
an appropriate "statistical microstate" can occur. Here, W stands for a number of all possi- 
ble (equiprobable) microstates that imply the prescribed macroscopic (e.g. thermodynamical) 
behavior corresponding to a fixed value of S. 

It is instructive to recall that if P is a probability of an event i.e. of a particular microstate, 
then — In P (actually, with log2 instead of In) may be interpreted as " a measure of information 
produced when one message is chosen from the set, all choices being equally likely" ("message" 
to be identified with a "microstate"). Another interpretation of — InP is that of a degree of 
uncertainty in the trial experiment, [Zj. 

As a pedestrian illustration let us invoke a classic example of a molecular gas in a box which 
is divided into two halves denoted "1" and "2". We allow the molecules to be in one of two 
elementary states: Ai if a molecule can be found in "1" half-box and A2 if it placed in another 



Let us consider a particular n-th macrostate of a molecular gas comprising a total of G molecules 
in a box, with n molecules in the state Ai and G — n molecules in the state 
The total number of ways in which G molecules can be distributed between two halves of the box 
in this prescribed macrostate, i.e. the number W = W{n) of distinct equiprobable microstates, 
clearly is W{n) = G\/[n\{G — n)\]. Here, P{n) = 1/W{n) is a probability with which any of 
microstates may occur in a system bound to "live" in a given macrostate. The maximum of W{n) 
and thus of kBlB.W{n) corresponds to Ni = N2 = n. 

To get a better insight into the information-uncertainty intertwine, let us consider an ensemble of 
finite systems which are allowed to appear in any of > distinct elementary states. The meaning 
of "state" is left unspecified, although an "alphabet" letter may be invoked for convenience. 
Let us pick up randomly a large sample composed of G ^ 1 single systems, each one in a 
certain (randomly assigned) state. We record frequencies rii/G = pi, ...jU^/G = with which 
the elementary states of the type 1,...,A^ do actually occur. This sample is a substitute for a 
"message" or a "statistical microstate" in the previous discussion. 

Next, we identify the number of all possible samples of that fixed size G which would show up 
the very same statistics pi, ...,PAf of elementary states. We interpret those samples to display the 
same "macroscopic behavior". 

It was the major discovery due to Boltzmann, see e.g. |3], that the number W of relevant "mi- 
croscopic states" can be approximately read out from each single sample and is directly related 
to the the introduced a priori probability measure /ii, ...,fiN, with an identification pi = Hi for all 
1 < i < A^, by the Shannon formula: 



half "2". 



N 




8 



1=1 



On the basis of this formula, we can consistently introduce S (/i) as the mean information per each 
(i-th) elementary state of the A^-state system, as encoded in a given sample whose size G ^ 1 is 
sufficiently large, p. 
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To exemplify previous considerations, let us consider N = 2. It is instructive to compare the 
uncertainty level (alternatively - information content) of S{fi) for the two-state system, if we take 
2 as the logarithm base instead of e. Then, Eq. (jSJ would refer to the binary encoding of the 
message (string) with G entries. 

We find that = 0.1 and /ig = 0.9 yield S{^) = 0.469. Analogously 0.2 and 0.8 imply 0.7219, 
while 0.3 and 0.7 give 0.8813. Next, 0.4 and 0.6 imply 0.971, and we reach an obvious maximum 
S = 1 for yUi = yU2 = 0.5. An instructive example of the "dog-flea" model workings with G = 50 
fleas jumping back and forth between their "states of residence" on a dog "1" or dog "2", can be 
found in Ref. [SS]- Albeit, in a number of specific cases, an evolution of the Gibbs entropy may 
show up some surprises if the "entropy growth dogma" is uncritically accepted, see e.g. examples 
in jSSl inn] and the discussion of Refs. jSZl EB] ■ 

By pursuing the Shannon's communication theory track, we can as well identify states of the 
model system with "messages" (strings) of an arbitrary length G > which are entirely composed 
by means of the prescribed N "alphabet" entries (e.g. events or alphabet letters Aj with the 
previous probability measure /i). Then, Eq. (jH)) may be interpreted as a measure of information 
per alphabet letter, obtained after a particular message (string = state of the model system) has 
been received or measured, c.f. our discussion preceding Eq. (jH)). In this case, the Shannon en- 
tropy interpolates between a maximal information (one certain event) and a minimal information 
(uniform distribution), cf. Eq. (jH)). 

Remark 3: Any string containing G = 10.000 symbols which are randomly sampled from among 
equiprobable N = 27 alphabet letters, jH], stands for a concrete microstate. In view of /i = 1/27, 
a corresponding macrostate is described via Eqs. © and (jH)) in terms of the number S{fi) = 
— log2(l/27) ~ 4.76. Accordingly, logg = G ■ 5(/i) ^ 47.600, where W is the number of 
admissible microstates. 

2.3 Shannon entropy and differential entropy 
2.3.1 Bernoulli scheme and normal distribution 

Let us consider again a two-state system where Ai appears with a probability fii = p while A2 
with a probability /i2 = 1 — p. A probability with which Ai would have appeared exactly n times, 
in the series of G repetitions of the two-state experiment, is given by the Bernoulli formula: 

^' p'^(l-p)^-" (9) 



" [n\{G-n 

where, in view of the Newton formula for the binomial {p + g)*^, after setting q = 1 — p we arrive 

Since the number n of successes in the Bernoulli scheme is restricted only by inequalities < 
n < G, what we have actually defined is a probability measure fi = {Pq, Pi, Pq} for G distinct 
random events denoted Bq, Bi, Bq- Accordingly, we can introduce a random variable B and 
say that it has the Bernoulli distribution, if B takes values n = 0, 1,...,G with the Bernoulli 
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probabilities Pn of Eq. (jHl) for all n. A random event Bn is interpreted as "taking the value n in 
the Bernoulli scheme". 

Let us denote P{B = k) = P{Bi) = Pk- We know that P{B < n) = Ylk<n'^k- The mean value 
E{B) of B reads E{B) = E?=o^ " = Gp. The variance E{[B - E{B)f) of B equals Gp{l-p). 
The local de Moivre-Laplace theorem tells us that for large values of G the binomial distribution 
can be approximated by the normal (Gauss) one: 

^ 1 ( {n-Gpf 

" " [2TxGp{l - p)]i/2 ''''P 2Gp(l - p) 

At this point we shall take an inspiration from Ref. [Zl] and relate the Bernoulli "success" prob- 
abilities with probabilities of locating a particle in an arbitrary interval on a line R. Namely, let 
us first consider an interval of length L: [0, L\ C R. Let G ^ 1, we define an interval grating unit 
to be r = L/G and next redefine Eq. (fTUj) to arrive at a probability per bin of length r <^ 1: 

-Pn = pM = exp J (11) 

with: Xn = nr, xq = Gpr and = Gr'^pil —p). Obviously, p{xn) is not a probability on its own, 
while r ■ p(x„) = Pn is a probability to find a particle in the n-th interval of length r out of the 
admitted number G = L/r of bins. 

For convenience let us specify p = 1/2 which implies xq = rG/2 and a = r^/G/2. We recall 
that almost all of the probability "mass" of the Gauss distribution is contained in the interval 
—3(7 < a^o < +3cr about the mean value Xq. Indeed, we have prob{\x — Xo\) < 2a) = 0.954 while 
prob{\x - Xo\) < 3a) = 0.998. 

The lower bound 100 < n < G justifies the usage of simplified versions of the standard Stirling 
formula n"' exp{—n)^/2^Tn < n\ < n"exp[— n -|- (1/I2n)]v27m, in view of the above probability 
"mass" estimates. Therefore, we can safely replace the Bernoulli probability measure by its (still 
discrete) Gaussian approximation Eq. (|TO|l and next pass to Eq. (|TT|l and its obvious continuous 
generalization. 

Remark 4: By taking a concrete input of L = 1 and G = 10^ we get the grating (spacing, 
resolution) unit r = 10~^. Then, xq = 1/2 while a = (1/2) ■ 10~^. It is thus a localization interval 
[1/2 - 3a, 1/2 + 3a] of length 6a = 3 ■ IQ-^ to be compared with L = 1. By setting G = 10^ we 
would get 6a = 3 ■ 10~^. 

For future reference, let us stress that generally we expect r ^ cr which implies a sharp distinction 
between the grating (resolution) unit r and the localization properties of the Gauss function 
expressed through its half-width a. 

2.3.2 Coarse-graining 

For a given probability density function on R we can adopt the coarse-graining procedure, 
giving account of an imprecision with which a spatial position x can be measured or estimated. 
Thus, if compared with the previous Bernoulli —>■ Gauss argument, we shall proceed in reverse from 
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density functions to approximating them by piece- wise constant, histogram- type discontinuous 
functions. 

We need to partition the configuration space R into a family of disjoint subsets (intervals) {-Bfc} 
such that UkBk C R and Bi fl Bj =0 for i ^ j. We denote fi{Bk) = fik the length of the fc-th 
interval, where n stands for the Lebesgue measure on R. 

A probability that a Gaussian random variable with the density p takes its value x in an interval Bk 
equals prob{Bk) = Pk = /g^ p{x)dx. An average of the density p over B^ we denote < p >k= Pk/Pk 
where pk = /g^ dx. 

The probability density p coarse grained with respect to the partition {Bk} reads: 

Pb{x) = ^< p>k lk{x) (12) 

k 

where Ifc(x) is an indicator (characteristic) function of the set Bk, which is equal 1 for x G -B^ and 
vanishes otherwise. Since J lk{x)dx = pk it is clear that 

/ pB{x)dx = ^ < p >k Pk = ^Pk = 'i- (13) 

k k 

where an interchange of the summation with integration is presumed to be allowed. 

By invoking arguments of the previous subsection, we choose a grating unit /x^ = r ^ 1 for all k 

and notice that < p >k= Pk/r where pk — p{xk) ■ r for certain Xk G Bk- 

In view of the twice "triple half- width" spatial localization property of the Gauss function, we 
can safely assert that an interval L ~ 6cr about Xq may be used in the coarse graining procedure, 
instead of the full configuration space R. Effectively, we arrive at a finite partition on L with 
the resolution L/G = r and then we can safely invoke the definition of = = r ■ p{xk), in 
conformity with Eq. (fTT|l . 

For a coarse grained probability density we introduce a coarse grained Shannon entropy whose 
relationship to the original differential entropy is of major interest. We have: 

'5(pb) = -^Pfclnpfc ~ (14) 

k 

- ^[i"p{xk)] Inr - ^[rp{xk)] \n[p{xk)] 

k k 

with a standard interpretation of the mean information per bin of length r. Here, if a partition 
(grating) unit r is small, one arrives at an approximate formula (we admit | lnr| ^ 1): 

5(pb) — — Inr — y p{x)\n[p{x)]dx = —Inr + S{p) (15) 

with the obvious proviso that S{pb) > and hence, in view of S{p) > Inr, we need to have 
maintained a proper balance between a and the chosen grating level r. 

Remark 5: It is instructive to validate the above approximation for the choice of r = 10~^, 
hence Inr = -6 In 10 13.86 . We have: S{p) = (1/2) ln(27rea2) ~ 0.92 + \na. By setting 
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cr = (1/2)10~^ we realize that S{p) ~ 0.92 — ln2 — 31nl0 ~ —6.7 hence is almost twice larger 
than the allowed lower bound Inr. The approximate value of the coarse grained entropy is here 
<S{pb) ~ 7.16 and stands for the mean information per partition bin in the "string" composed of 
G bins. 

In view of Eq. fll5|) . as long as we keep in memory the strictly positive grating unit r, there is 
a well defined " regularization" procedure (add — Inr to S{p)) which relates the coarse grained 
entropy with a given differential entropy. In a number of cases it is computationally simpler to 
evaluate the differential entropy, and then to extract - otherwise computationally intractable - 
coarse grained entropy. 

Notice that one cannot allow a naive zero grating limit in Eq. fllSp . although r may be arbitrarily 
small. Indeed, for e.g. the Gaussian densities, differential entropy takes finite values and this 
would suggest that the coarse grained entropy might be arbitrarily large. This obstacle does not 
arise if one proceeds with some care. One should remember that we infer the coarse grained 
entropy from the differential entropy exactly at the price of introducing the resolution unit r. The 
smaller is r, the better is an approximation of the differential entropy by the second term on the 
right-hand-side of Eq. ()15|1 , but — In r needs to remain as a finite entry in Eq. ()15|1 . 
We have inequalities < S{pb) < G where L = G ■ r. They extend to all approximately equal 
entries in Eq. (|T5|l . Since — Inr = — InL -|- InG, we arrive at new inequalities: 

Inr < - ^[rp(xfc)] ln[p(xfc)] < InL (16) 

k 

where ^i^[rp{xk)]\T^^[p{xk)] =^ — J p^npdx with r — > and possibly L —>■ oo. A conclusion is 
that the differential entropy is unbounded both form below and from the above. In particular, 
S{p) may take arbitrarily low negative values, in plain contrast to its coarse grained version S{pb) 
which is always nonnegative. 

We can be more detailed in connection with approximations employed in Eqs. (16) and (fTH|) . 
Actually, the right-hand-side of Eq. (fT3j) sets a lower bound for the coarse-grained entropy S{pb)- 
Let us recall that the value of a convex function xlnx at the mean value of its argument (x) , does 
not exceed the mean value {x In x) of the function itself. Then, in our notation which follows Eq. 
(12), we can directly employ an averaging over 2?^,: 



p In pdx > i - pdx 



In I - / pdx 



(17) 



Taking the minus sign, executing summations with respect to k (convergence of the series being 
presumed) and using Eqs. (15) and (16) we get: 

S{p) - In r < S{pb) (18) 

complement to Eq. (17), see e.g. also j2Zll22]- 
Equations (16) and ()18p allow, with suitable reservations, to extend the standard information/uncertainty 
measure meaning from coarse-grained entropies to differential entropies per se. Namely, the differ- 
ence of two coarse grained entropies, corresponding to the same partition but to different (coarse 



Entropy 2005, 7[4], 253-299 



16 



grained) densities, may be adequately approximated by the difference of the corresponding differ- 
ential entropies: 

S{pb) - S{p'^) c:. S{p) - S{p') , (19) 
provided they take finite values, [01 EI! • 

2.3.3 Coarse-graining exemplified: exponential density 

An exponential density on a positive half-line R'^ is known to maximize a differential entropy 
among all R'^ density functions with the first moment fixed at 1/A > 0. The density has the form: 
p{x) = Aexp(— Ax) for a; > and vanishes for a; < 0. Its variance is 1/A^. The differential entropy 
of the exponential density reads S{p) = 1 — In A. 

In he notation of the previous subsection, let us coarse-grain this density at a particular value of 
A = 1 and then evaluate the corresponding entropy as follows. We choose r <^ 1, — p{xk) ■ r 
with Xk = kr where is a natural number. One can directly verify that for small r, we can write 
r ~ 1 — exp{—r) and thence consider ~ [1 — exp{—r)] exp(— fcr), such that X^fcLo^**-' ~ '^ith 
the well known quantum mechanical connotation. 

Namely, let us set r = /ii^/fcsT; notice that for the first time in the present paper, we explicitly 
invoke dimensional units, in terms of which the dimensionless constant r is defined. We readily 
arrive at the probability of the ku-th oscillator mode in thermal bath at the temperature T. 
Our assumption of r <^ 1 corresponds to the low frequency oscillator problem with z/ <^ 1, see 
e.g. jH]. Clearly, we have for the mean number of modes 

(n) = Vfcpfc = (20) 

^-^ exp r — 1 r 

which implies the familiar Planck formula 

{E) = ^ ^ hu/r = ksT . (21) 



For the variance we get 



1 



{{n-{n)y) = {ny~{n)^-. (22) 



r 



The standard Shannon entropy of the discrete probability distribution pk,k & N reads 

Sips) = -^Pfclnpfc = -ln[l -exp(-r)] + {n)r (23) 

so that in view of S{pb) ~ — Inr -|- 1 and S{p) = 1 for p(x) = exp(— x), we clearly have S{pb) — 
— \nr + S{p), as expected. 

Let us point out our quite redundant sophistication. In fact, we could have skipped all the above 
reasoning and take Eq. (fTH|l as the starting point to evaluate S{pb) for the coarse grained 
exponential density, at the assumed resolution r = hu/kBT <^ 1, with the obvious result 



S{pb) ^ 1 - Hhu/kBT) > 1 



(24) 
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Remark 6: An analogous procedure can be readily adopted to analyze phenomenological his- 
tograms of energy spectra for complex systems, with an obvious extension of its range of validity 
to spectral properties of the classically chaotic case. In particular, semiclassical spectral series 
of various quantum systems admit an approximation of spacing histograms by continuous distri- 
butions on the positive half-line R^. Although a full fledged analysis is quite complicated, one 
may invoke quite useful, albeit approximate formulas for adjacent level spacing distributions. The 
previously mentioned exponential density corresponds to the so-called Poisson spectral series. In 
the family of radial densities of the form Pn{x) = p^^|^x^~^exp(— x^) where > 1 and T is 
the Euler gamma function, the particular cases A^ = 2, 3, 5 correspond to the generic level 
spacing distributions, based on the exploitation of the Wigner surmise. The respective histograms 
plus their continuous density interpolations are often reproduced in "quantum chaos" papers, see 
for example jHU] . 

2.3.4 Spatial coarse graining in quantum mechanics 

The coarse grained entropy attributes the "mean information per bin of length r" to systems 
described by continuous probability densities and their differential entropies. Effectively one has a 
tool which allows to accompany the coarse grained density histogram (of pk in the k-th bin on R) 
by the related histogram of uncertainties — Inp^, c.f. Section II. A where an uncertainty function 
has been introduced. 

The archetypal example of position measurement in quantum mechanics presumes that position is 
measured in bins corresponding to the resolution of the measurement apparatus. This means that 
the continuous spectrum of the position observable is partitioned into a countable set of intervals 
(bins) whose maximum length we regard as a "resolution unit". For an interval C R we may 
denote pk the probability of finding the outcome of a position measurement to have a value in B^. 
We are free set the bin size arbitrarily, especially if computer assisted procedures are employed. 

Hi. 

Following one may take the view that the most natural measure of the uncertainty in the 
result of a position measurement or preparation of specific localization features of a state vector 
(amenable to an analysis via spectral properties of the position operator) should be the information 
entropy coming from Eqs. (fT^ and (fT!?jl : S{pb) = — "^kPk^^Pk with a direct quantum input 
Pk = \'ip{x)\'^dx, where ip G L'^{R) is normalized. This viewpoint is validated by current 
experimental techniques in the domain of matter wave interferometry, |S1 HB] , and the associated 
numerical experimentation where various histograms are generated, [75] . 

The formula Eq. (fTH|) gives meaning to the intertwine of the differential and coarse grained en- 
tropies in the quantum mechanical context. When an analytic form of the entropy is in the reach, 
the coarse graining is straightforward. One should realize that most of the results known to 
date have been obtained numerically, hence with an implicit coarse-graining, although they were 
interpreted in terms of the differential entropy, see e.g. [2H]-|SZ1- 

In connection with an entropic inequality Eq. (j3]) let us point out |2] that it is a generic property 
of normalized L^(i?") wave functions that, by means of the Fourier transformation, they give rise 



Entropy 2005, 7[4], 253-299 



18 



to two interrelated densities (presently we refer to L'^{R)): p = {ipl'^ and p = where 

{J-'ip){k) = I ip{x) exp{—ikx) dx (25) 

is the Fourier transform of ip{x). The inequality (0} for the corresponding (finite) differential 
entropies follows, here with n = 1. 

By choosing resolutions r <^ 1 and f ^ 1 we can introduce the respective coarse grained entropies, 
each fulfilling an inequality Eq. (fT^ . Combining these inequalities with Eq. (jlj), we get the 
prototype entropic inequalities for coarse grained entropies: 

S{pb) + S{pb) > 1 + In tt - ln(r ■ f ) (26) 

with the corresponding resolutions r and f. 

By referring to Eq. (fTHjl we realize that the knowledge of iS(ps), completely determines S{pb) at 
the presumed resolution levels: 

5(pb) ~ 1 + ln7r-ln(r -f) -5(pb) > (27) 

and in reverse. This in turn implies that in all computer generated position-momentum differential 
entropy inequalities, where the coarse graining is implicit, the knowledge of position entropy and of 
the resolution levels provide sufficient data to deduce the combined position-momentum outcomes, 
see also [Hl-EZI- 

In standard units (with h reintroduced), the previous discussion pertains to quantum mechanical 
position - momentum entropic uncertainty relations. In the notation of Refs. ^] and [22] we 
have: 

' 6x ■ 6p^ 



S"" + >l-\n2-\n\^—^j (28) 

for measurement entropies with position and momentum "abstract measuring device" resolutions 
6x and 6p respectively, such that 6x ■ 6p h. 

Note, that to reintroduce h we must explain how the differential entropy definition is modified if 
we pass to dimensional units, see e.g next subsection. Let us also point out that one should not 
confuse the above resolution units r, f and 6x, 6p with standard mean square deviation values AX 
and AP which are present in the canonical indeterminacy relations: AX ■ AP > h/2. 
Let us set h= 1 (natural units system). If, following conventions we define the squared standard 
deviation (i.e. variance) value for an observable A in a pure state ip as (AA)"^ = {ip, [A — {A)]'^ip) 
with (A) = [ip , Aijj) , then for the position X and momentum P operators we have the following 
version of the entropic uncertainty relation (here expressed through so-called entropy powers, see 
e.g. 0): 

AX ■ AP > ^ exp[5(p) + S{p)] > \ (29) 
which is an alternative for Eq. (Q; = 1 being implicit. 
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2.4 Impact of dimensional units 

Let us come back to an issue of reintroducing physical units in Eq. (j2Hl)- In fact, if x and 
p stand for one-dimensional phase space labels and f{x,p) is a normalized phase-space density, 
/ f{x,p)dxdp = 1, then the related dimensionless differential entropy is defined as follows, pTHliO]: 

Sh = - f {hf)ln{hf)^ = - [ fln{hf)dxdp (30) 



h 

where h = 2Trh is the Planck constant. If p{x) = \4'\'^{x), where ip ^ -^^(-R); then ph{p) = 
\!Fh{ilj)\'^{p) is defined in terms of the dimensional Fourier transform: 

1 f 

{Thip){p) = —j= ip{x) exp{—ipx/fi) dx . (31) 



V2Trh 

Let us consider the joint density 

f{x,p) = p{x)ph{p) (32) 

and evaluate the differential entropy Sh for this density. Remembering that J p{x)dx = 1 = 
/ Ph{p)dp, we have formally, [10] Eq. (9): 

Sh = — plnpdx— / ph\n phdp — \nh = + — \nh (33) 



to be compared with Eq. (f^H|l . The formal use of the logarithm properties before executing 

integrations in the / ph\^{hph) dp, has left us with the previously mentioned issue (Section 1.2) 

of "literally taking the logarithm of a dimensional argument" i. e. that of \nh. 

We recall that Sh is a dimensionless quantity, while if x has dimensions of length, then the 

probability density has dimensions of inverse length and analogously in connection with momentum 

dimensions. 

Let us denote x = r6x and p = rSp where labels r and f are dimensionless, while 6x and 6p stand 
for respective position and momentum dimensional (hitherto - resolution) units. Then: 



— J p\n pdx — \n{6x) = — j p\n{6xp)dx (34) 
is a dimensionless quantity. Analogously 

- / phln ph dp- In 6p = - I phln{6pph)dp (35) 



is dimensionless. First left-hand-side terms in two above equations we recognize as and 
respectively. 

Hence, formally we get a manifestly dimensionless decomposition 

f f SxSp SxSp 

Sh = - pln{6xp)dx - / ph \n{6pph) dp + In — — = Sg^ + 5? + In — — (36) 



instead of the previous one, Eq. (jHHjl . The last identity Eq. gives an unambiguous meaning 
to the preceding formal manipulations with dimensional quantities. 
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As a byproduct of our discussion, we have resolved the case of the spatially interpreted real axis, 
when X has dimensions of length, c.f. also jTHj: S^^ is the pertinent dimensionless differential 
entropy definition for spatial probability densities. 

Example 1: Let us go through an explicit example involving the Gauss density 



p(x) = (l/(TV27r) exp[-(a; - Xoy/2a'] (37) 

where cr is the standard deviation (its square stands for the variance). There holds S{p) = 
I In (27recT^) which is a dimensionless outcome. If we pass to x with dimensions of length, then 
inevitably a must have dimensions of length. It is instructive to check that in this dimensional 
case we have a correct dimensionless result: 

5L = ^ln[27re(£)'] (38) 

to be compared with Eq. (jHH). Then we realize that S^^ vanishes if a/Sx = (27re)~^''^, hence at 
the dimensional value of the standard deviation a = (27re)^^/^(5x, compare e.g. [TUj . 
Example 2: Let us invoke the simplest (naive) text-book version of the Boltzmann H-theorem, 
valid in case of the rarified gas (of mass m particles), without external forces, close to its thermal 
equilibrium, under an assumption of its space homogeneity, |SZ1 ESj- If the probability density 
function f{v) is a solution of the corresponding Boltzmann kinetic equation, then the Boltzmann 
i7-functional (which is simply the negative of the differential entropy) H{t) = J f{v) In f{v)dv does 
not increase: ^H{t) < 0. In the present case we know that there exists an invariant (asymptotic) 
density, which in one-dimensional case has the form /*(f ) = {m/27ikBTy^'^ exp[— m(f — f o)^/2A;b^]- 
H{t) is known to be time-independent only if / = f*{v). We can straightforwardly, albeit for- 
mally, evaluate = j f^hif^^dv = — (1/2) ln(27reA;sT/m) and become faced with an apparent 
dimensional difficulty, [Hj, since an argument of the logarithm is not dimensionless. For sure, 
a consistent integration outcome for H{t) should involve a dimensionless argument A;BT/m[f]^ 
instead of ksT/m, provided [v] stands for any unit of velocity; examples are [v] = Im/s (here 
m stands for the SI length unit, and not for a mass parameter) or 10~^m/s. To this end, in 
conformity with our previous discussion, it suffices to redefine as follows, |^ llOj: 

H,^H'^:^= j UH[v]-U)dv. (39) 

Multiplying by [v\ we arrive at the dimensionless argument of the logarithm in the above and 
cure the dimensional obstacle. 

Remark 7: Let us also mention an effect of the scaling transformation upon the differential 
entropy, We denote 

Pa,/3 = /5p[/3(x-a)] (40) 
where a > 0, /3 > 0. The respective Shannon entropy reads: 

S{p^,p)=S{p)-\n(3. (41) 



In case of the Gaussian p, Eq. (jTfjl . we get S{pa,f3) = ln[(cr//3) v27re]. An obvious interpretation 
is that the /3-scaling transformation of p{x — a) would broaden this density if /3 < 1 and would 
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shrink when (3 > 1. Clearly, S{pa,f3) takes the value at a = (27re)~^/^/5 in analogy with our 
previous dimensional considerations. If an argument of p is assumed to have dimensions, then 
the scaling transformation with the dimensional j3 may be interpreted as a method to restore the 
dimensionless differential entropy value. 

3 Localization: differential entropy and Fisher information 

We recall that among all one-dimensional distribution functions p{x) with a finite mean, subject 
to the constraint that the standard deviation is fixed at a, it is the Gauss function with half- 
width a which sets a maximum of the differential entropy, For the record, let us add that if 
only the mean is given for probability density functions on R, then there is no maximum entropy 
distribution in their set. 

Let us consider the Gaussian probability density on the real line i? as a reference density function: 
p{x) = (l/cTA/27r) exp[— (x — Xo)^/2cr^], in conformity with Eq. (fTTj). but without any restriction 
on the value of x G -R. 

The differential entropy of the Gauss density has a simple analytic form, independent of the mean 
value Xo and maximizes an inequality: 



with an obvious bearing on the spatial localization of the density p, hence spatial (un) certainty of 
position measurements. We can say that almost surely, with probability 0.998, there is nothing 
to be found (measured) beyond the interval of the length 6a which is centered about the mean 
value Xq of the Gaussian density p. 

Knowing that for arbitrary density functions the differential entropy Eq. (jH)) is unbounded form 
below and from above, we realize that in the subset of all densities with a finite mean and a 
fixed variance cr^, we actually have an upper bound set by Eq. (jl^ . However, in contrast to 
coarse grained entropies which are always nonnegative, even for relatively large mean deviation 
a < l/v27re ~ 0.26 the differential entropy S{p) is negative. 

Therefore, quite apart from the previously discussed direct information theory links, c.f. Eqs. (fTH|) . 
()18|) and (fT^ . the major role of the differential entropy is to be a measure of localization in the 
"state space" (actually, configuration space) of the system, [ZHl EEl EZ! • 

Let us consider a one-parameter family of probability densities pa{x) on R whose first (mean) 
and second moments (effectively, the variance) are finite. The parameter-dependence is here not 
completely arbitrary and we assume standard regularity properties that allow to differentiate 
various functions of pa with respect to the parameter a under the sign of an (improper) integral. 




(42) 



This imposes a useful bound upon the entropy power: 



1 



exp[5(p)] < a 



(43) 



V2^ 
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Namely, let us denote / xpa{x)dx = f{a) and / x^padx < oo. We demand that as a function of 
X E R, the modulus of the partial derivative dpa/da is bounded by a function G{x) which together 
with xG{x) is integrable on R. This implies, jH2], the existence of df /da and an important 
inequality: 



\ da ) \ da 



directly resulting from 



«)V„<;-/f^Vp„<;.>f^i (44) 



(45) 



via the standard Schwarz inequality. An equality appears in Eq. ()44|1 if pa{x) is the Gauss function 
with mean value a. 

At this point let assume that the mean value of pa actually equals a and we fix at cr^ the value 
((x — aY) = (x^) — «^ of the variance (in fact, standard deviation from the mean value ) of the 
probability density p^. The previous inequality Eq. ()44|) now takes the familiar form: 

where an integral on the left-hand-side is the so-called Fisher information of pa, known to ap- 
pear in various problems of statistical estimation theory, as well as an ingredient of a number of 
information-theoretic inequalities, jH2l 123 1211 ESI E21 ■ In view of JF^, > l/a^, we realize that the 
Fisher information is more sensitive indicator of the wave packet localization than the entropy 
power, Eq. (jlHj) . 

Let us define Pq{x) = p{x — a). Then, the Fisher information J-'a = is no longer the mean value 
a-dependent and can be readily transformed to the conspicuously quantum mechanical form (up 
to a factor with D = h/2m): 

where m = V In p is named an osmotic velocity field, [SUl EH] , and an average (Q) = J p ■ Qdx is 
carried out with respect to the function 

Q = 2^ . (48) 

As a consequence of Eq. (jlEI), we have —{Q) > l/2cr^ for all relevant probability densities with 
any finite mean and variance fixed at cr^. 

When multiplied by D^, the above expression for Q{x) notoriously appears in the hydrodynamical 
formalism of quantum mechanics as the so-called de Broglie-Bohm quantum potential {D = h/2m). 
It appears as well in the corresponding formalism for diffusion-type processes, including the stan- 
dard Brownian motion (then, D = ksT/mP, see e.g. I54 | IMj . 

An important inequality, valid under an assumption Pa{x) = p{x — a), has been proved in 
see also j^lHK]: 

^ < (27re) exp[-25(p)] < ^ (49) 
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It tells us that the lower bound for the Fisher information is in fact given a sharper form by means 
of the (squared) inverse entropy power. Our two information measures appear to be correlated. 

Remark 8: Let us point out that the Fisher information J-'{p) may blow up to infinity under 
a number of circumstances, jHS]: when p approaches the Dirac delta behavior, if p vanishes over 
some interval in R or is discontinuous. We observe that > because it may vanish only when 
p is constant everywhere on R, hence when p is not a probability density. 

Remark 9: The values of J-'{pa) and S{pa) are a-independent if we consider pa{x) = p{x — a). 
This reflects the translational invariance of the Fisher and Shannon information measures, [5^ . 
The scaling transformation pa^p = [3 p[[3{x — a)], where a > 0,/3 > 0, transforms Eq. to the 
form (27re)^i/2 exp[5(p«,/3)] < cr/(3. 

Under an additional decomposition/factorization ansatz (of the quantum mechanical L?'{R^) 
provenance) that p{x) = \ip\'^{x), where a real or complex function ijj = y^exp{i(f)) is a nor- 
malized element of L'^{R), another important inequality holds true, [2312]: 

^ = 4/(^)'rfx<16vrV, (50) 

provided the Fisher information takes finite values. Here, is the variance of the "quantum 
mechanical momentum canonically conjugate to the position observable", up to (skipped) dimen- 
sional factors. In the above, we have exploited the Fourier transform ip = (J^ip) of to arrive at 
p = of Eq. (jU whose variance the above cr^ actually is. 

In view of two previous inequalities ()49|) . (j^UI) we find that not only the Fisher information, but 
also an entropy power is bounded from below and above. Namely, we have: 

^ < < IGn^a^ (51) 

which implies l/2o"^ < —{Q) < Svr^cr^ and furthermore 

-L<^exp[5(p)]<a. (52) 
Ana V27re 



as a complement to Eq. ()43|1 . Most important outcome of Eq. ()52jl is that the differential entropy 
S{p) typically may be expected to be a well behaved quantity: with finite both lower and upper 
bounds. 

We find rather interesting that the Heisenberg indeterminacy relationship Eq. ()29|) . which is 
normally interpreted to set a lower bound on the experimentally accessible phase-space data (e.g. 
volume), according to Eq. (jK^ ultimately had appeared to give rise to lower and upper bounds 
upon the configurational (spatial) information measure and thence - the uncertainty (information) 
measure. 

To our knowledge, the inequalities Eqs. ()5H1 and ()52|1 . although implicit in various information 
theory papers, see especially [2S1 and [2], hitherto were never explicitly spelled out. 
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4 Asymptotic approach towards equilibrium: Smoluchowski processes 

4.1 Random walk 

Let us consider a classic example of a one-dimensional random walk where a particle is assumed 
to be displaced along with a probability 1/2 forth and back, each step being of a unit length, 
|74j . If one begins from the origin 0, after G steps a particle can found at any of the points 
—G, —G -|- 1, ... — 1, 0, 1, G. The probability that after G displacements a particle can be found 
at the point g G [—G, G] is given by the Bernoulli distribution: 



G\ /I 

[(1/2)(G + ^)]![(1/2)(G-^)]! U 



= , ( ) (53) 



G 



where Y.%_GPg = 1- 

We are interested in the asymptotic formula, valid for large G and g <^ G. (Note that even for 
relatively small value of G = 20, and \g\ < 16, an accuracy level is satisfactory.) There holds: 



and, accordingly 



n-(4)'"exp(-^). (55) 



We assume r ~ 10~^m, to be a grating unit (i.e. minimal step length for a walk). Let r <^ Ax ^ G 
(size Ax ~ 10~^m is quite satisfactory). For large G and \g\ <^ G, we denote x = g-r and ask for a 
probability pg{x)Ax that a particle can be found in the interval [x, x-l- Ax] after G displacements. 
The resuh is tZ4j: 

1 1 f x'^ \ 

Pi^) = = /o„r^^2M/2 -7T77-^ (56) 



2r (27rG'r2)i/2 ^ ^ 2Gr\ 

and by assuming that a particle suffers displacements per unit time, we can give Eq. (j^Ujl the 
familiar form of the heat kernel: 

^) = /. ,^.M/9 exp (-^^ (57) 



(47rDt)i/2 - 4Dt 

with the diffusion coefficient D = kr'^/2. It is a fundamental solution of the heat equation 
dtp = DAp which is the Fokker-Planck equation for the Wiener process. 
The differential entropy of the above time- dependent density reads: 

S{t) = (1/2) ln{AneDt) (58) 

and its time evolution clearly displays the localization uncertainty growth. By means of the 
formula Eq. ()19p we can quantify the differential entropy dynamics for all solutions of the heat 
equation. 

Since the heat kernel determines the transition probability density for the Wiener process (free 
Brownian motion in R), by setting x — > x — x' and i — > t — t' > 0, we can replace the previous 
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p{x, t) of Eq. (j57j) by p{x — x',t — t'). This transition density allows to deduce any given solution 
t) of the heat equation from its past data, according to: p(x, t) = f p{x — x', t — t')p{x', t, )dx' . 
In particular, we can consider the process starting at t' = with any initial density Pq{x). 
Let pv denote a convolution of a probability density p with a Gaussian probability density having 
variance v. The transition density of the Wiener process generates such a convolution for p^, with 
V = a"^ = 2Dt. Then, de Bruijn identity, jlHllZHl, dS{pv)/dv = (l/2)jF(p„), directly yields the 
information entropy time rate for S{p) = S{t): 



The Fisher information J^{p) is the a = version of the general definition given in Eqs. ()46p and 
(jlZj). The derivation of Eq. (jSHI) amounts to differentiating an ^-dependent integrand under the 
sign of an improper integral, [82, 83]) 

The monotonic growth of S{t) is paralleled by linear in time growth of a{t) and the decay of JF, 
hence quantifies the uncertainty (disorder) increase related to the "flattening" down of p, see also 



4.2 KuUback entropy versus differential entropy 

We emphasize that in the present paper we have deliberately avoided the use of the relative 
Kullback-Leibler entropy, [HU El EZI- This entropy notion is often invoked to tell "how far from 
each other" two probability qdensities are. The KuUback entropy is particularly useful if one 
investigates an approach of the system toward (or its deviation from) equilibrium, this being 
normally represented by a stationary density function, |?H1 |SH]- In this context, it is employed 
to investigate a major issue of the dynamical origins of the increasing entropy, see |T7t EH| 102] • 
Consult also both standard motivations and apparent problems encountered in connection with 
the celebrated Boltzmann's if-theorem, [HZIESI and |67j . 

However, a reliability of the Kullback entropy may be questioned in case of general parameter- 
dependent densities. In particular, this entropy fails to quantify properly certain features of a 
non-stationary dynamics of probability densities. Specifically if we wish to make a "comparison" 
of once given density function to itself, but at different stages (instants) of its time evolution. 
Let us consider a one parameter family of Gaussian densities pa = p{x — a), with the mean a E R 
and the standard deviation fixed at cr. These densities are not differentiated by the information 
(differential) entropy and share its very same value iSo- = | In (27re(j^) independent of a. 
If we admit a to be another free parameter, a two-parameter family of Gaussian densities pa — ^ 
Pa,a{x) appears. Such densities, corresponding to different values of ex and a' do admit an "absolute 
comparison" in terms of the Shannon entropy, in accordance with Eq. p9|l : 



By denoting a = a{t) = y/2Dt and a' = (j{t') we make the non- stationary (heat kernel) density 
amenable to the "absolute comparison" formula at different time instants t' > t > 0: (cr'/cr) = 




(59) 



ES] 




(60) 
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In the above we have "compared" differential entropies of quite akin, albeit different, probability 
densities. Among many inequivalent ways to evaluate the "divergence" between probability dis- 
tributions, the relative (Kullback) entropy is typically used to quantify such divergence from the 
a priori prescribed reference density, [HHlllHl- 

We define the Kullback entropy }C{6, 9') for a one-parameter family of probability densities pe, so 
that the "distance" between any two densities in this family can be directly evaluated. Let pei 
stands for the prescribed (reference) probability density. We have, [HHISHllEl: 

1C{e, 6') = lC{pe\pe') = I Pe{x) In dx . (61) 

J Pe'[x) 

which, in view of the concavity of the function f{w) = —wlnw, is positive. 

Let us indicate that the negative of JC, He = — /C, named the conditional entropy [H], is predomi- 
nantly used in the literature jHl EZl HE] because of its affinity (regarded as a formal generalization) 
to the differential entropy. Then e.g. one investigates an approach of — /C towards its maximum 
(usually achieved at the value zero) when a running density is bound to have a unique stationary 
asymptotic, [02] • 

If we take 6' = 6 + A6 with A^^ <C 1, the following approximate formula holds true under a number 
of standard assumptions, jHI]: 

icie,e + Ae)^^Te-iAOf (62) 



where J-'g denotes the Fisher information measure, previously defined by Eq. (jloj). With this 
proviso, we can evaluate the Kullback distance within a two-parameter (a, a) family of Gaussian 
densities, by taking 9 ^ a. 

Passing to a' = a -|- Aa at a fixed value of a we arrive at: 

/C(a, a + Aa) ~ -^^^ . (63) 

For the record, we note that the respective Shannon entropies do coincide: Sa = Sa+Aa- 
Analogously, we can proceed with respect to the label a at a fixed: 

/C((T, a + Aa) ~ -^^^ (64) 

when, irrespective of a: 

Sa+Aa - 5^ =i • (65) 

a 

By choosing ^ cr^ at a fixed, we get (now the variance is modified by its increment A((t^)): 

/C(a^a^ + A(a^))^i^gl£ (66) 

while 

0(72+A(,72) - o„2 ~ „ (67) 
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which, upon identifications cr^ = 2Dt and A(o"^) = 2D At, sets an obvious connection with the 
differential {AS){t) and thence with the time derivative S = l/2t of the heat kernel differential 
entropy, Eq. and the de Bruijn identity. 

Our previous observations are a special case of more general reasoning. Namely, if we consider a 
two-parameter 6 = {6i, 62) family of densities, then instead of Eq. ()62|) we would have arrived at 



]C{e, ^ + ~ i ^ J^ij ■ AOiAej (6^ 



2 

where = 1, 2 and the Fisher information matrix J^ij has the form 

f d\npe d\npe , . 

In case of Gaussian densities, labelled by independent parameters 61 = a and 62 = c (alternatively 
62 = cr^), the Fisher matrix is diagonal and defined in terms of previous entries J-'a and JF^. (or 

It is useful to note (c.f. also [22]) that in self-explanatory notation, for two 9 and 6' Gaussian 
densities there holds: 

IC{e, = In - + ^(4 - 1) + A(« - (70) 
ex Z a Za' 

The first entry in Eq. (f7n|) coincides with the " absolute comparison formula" for Shannon entropies, 
Eq. (jnO))- However for |^^' — <^ 1, hence in the regime of interest for us, the second term dominates 
the first one. 

Indeed, let us set a' = a and consider cr^ = 2Dt, A{a'^) = 2D At. Then S{a') - S{a) ~ At/2t, 
while }C{6,6') (At)^/4t^. Although, for finite increments At we have 

S{a') - S{a) ^ ^/mO^) ^ ^ , (71) 

the time derivative notion S can be defined exclusively for the differential entropy, and is mean- 
ingless in terms of the Kullback "distance". 

Let us mention that no such obstacles arise in the standard cautious use of the relative Kullback 
entropy Ti^- Indeed, normally one of the involved densities stands for the stationary reference one 
pe'{x) = p*(x), while another evolves in time pei^x) = p{x,t), t G R~^, thence Hcit) = —}C{pt\p*), 
see e.g. 

4.3 Entropy dynamics in the Snioluchowski process 

We consider spatial Markov diffusion processes in R with a diffusion coefficient (constant or 
time-dependent) D and admit them to drive space-time inhomogeneous probability densities p = 
p{x,t). In the previous section we have addressed the special case of the free Brownian motion 
characterized by the current velocity (field) v = v{x, t) = —u{x, t) = DV In p{x, t) and the diffusion 
current j = v ■ p which obeys the continuity equation dtp = — Vj, this in turn being equivalent to 
the heat equation. 
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It is instructive to notice that the gradient of a potential-type function Q = Q{x,t), c.f. Eq. JUI), 
entirely composed in terms of u: 

Q = 2D^-^ = -u' + D\/-u (72) 

almost trivially appears (i.e. merely as a consequence of the heat equation, j^Hl Elj) in the 
hydrodynamical (momentum) conservation law appropriate for the free Brownian motion: 

dtv + {v ■ V)v = -VQ . (73) 

A straightforward generalization refers to a diffusive dynamics of a mass m particle in the external 
field of force, here taken to be conservative: F = F[x) = —W. The associated Smoluchowski 
diffusion process with a forward drift b{x) = ^ is analyzed in terms of the Fokker-Planck equation 
for the spatial probability density p(a;,t), |HH1 E3 EOl ED : 

dtp = DAp - V(6 ■ p) (74) 

with the initial data po{x) = p{x,0). 

Note that if things are specialized to the standard Brownian motion in an external force field, we 
know a priori (due to the Einstein fluctuation-dissipation relationship, [Zll) that D = where 
f3 is interpreted as the friction (damping) parameter, T is the temperature of the bath, ks being 
the Boltzmann constant . 

We assume, modulo restrictions upon drift function [S21inHI, to resolve the Smoluchowski dynam- 
ics in terms of (possibly non-unique) Markovian diffusion- type processes. Then, the following 
compatibility equations follow in the form of hydrodynamical conservation laws for the diffusion 
process, IMj : 

dtp + V{vp) = (75) 
{dt + v-V)v = V{n-Q) (76) 

where, not to confuse this notion with the previous force field potential V, we denote by Q{x) the 
so-called volume potential for the process: 

1 , „ S2 



"=2(^J ^''^■l^J' <"> 

where the functional form of Q is given by Eq. ()72|). Obviously the free Brownian law, Eq. (|73j). 
comes out as the special case. 

In the above (we use a short-hand notation v = v{x,t)): 

v = b-u = ^- (78) 

mjj p 

defines the current velocity of Brownian particles in external force field. This formula allows us 
to transform the continuity equation into the Fokker-Planck equation and back. 
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With a solution p{x,t) of the Fokker-Planck equation, we associate a differential (information) 
entropy S{t) = — J p In pdx which is typically not a conserved quantity, |Sn!-|2H]- The rate of 
change in time of S{t) readily follows. 

Boundary restrictions upon p, vp and bp to vanish at spatial infinities (or at finite spatial volume 
boundaries) yield the rate equation: 

f = /[p(V-6)+Z^-^]rf. (79) 
to be compared with the previous, 6 = case, Eq. (ffH|) . 

Anticipating further discussion, let us stress that even in case of plainly irreversible diffusive 
dynamics, it is by no means obvious that the differential entropy should grow, decay (diminish) 
or show up a mixed behavior. It is often tacitly assumed that one should "typically" have S > 
which is not true, '87]. 

We can rewrite Eq. ()79p in a number of equivalent forms, like e.g. (note that (m^) = —D{V ■ u)) 

DS = D{V -b) + (u^) = D{V -v) (80) 
and specifically, [HZlinHI, as the major entropy balance equation: 

DS = {v'^) - {b-v) = -{vu) (81) 
where (■) denotes the mean value with respect to p. 

This balance equation is extremely persuasive, since b = F/{mf3) and j = vp combine into a 
characteristic "power release" expression: 

^ = TT / ■3dx = ^{b-v) . (82) 

dt D J m(3 ' ^ ' 

Like in case of not necessarily positive S, the "power release" expression Q may be positive which 
represents the power removal to the environment, as well as negative which corresponds to the 
power absorption from the environment. 

In the formal thermodynamical lore, in the above we deal here with the time rate at which the 
mechanical work per unit of mass may possibly being dissipated (removed to the reservoir) in the 
form of heat, in the course of the Smoluchowski diffusion process: ksTQ = j F ■ j dx, with T 
being the temperature of the bath. When there is no external forces, we have 6 = 0, and then the 
differential entropy time rate formula for the free Brownian motion Eq. reappears. 
On the other hand, the positive terms in Eq. ()81|) and Eq. ()59|) represent the rate at which informa- 
tion entropy is put (pumped) into the diffusing system by the thermally active environment, thus 
causing a disorder/uncertainty growth. This particular "entropy production" rate may possibly 
be counterbalanced (to this end we need external forces) by the heat removal due to dissipation, 
according to: 

dS /dS\ dO , , 
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where Q is defined in Eq. §^ while {S)in = {1/D){v^). 

Remark 10: In Refs. |681 1671 lU^ a measure-theoretic and probabihstic justification was given to 
an interpretation of {1/D){v'^) as the entropy production rate of the (originally - stationary) diffu- 
sion process with the current velocity v. We would like to point out that traditionally, [TmiTH IBT]. 
the statistical mechanical notion of an entropy production refers to the excess entropy that is 
pumped out of the system. An alternative statement tells about the entropy production by the 
physical system into the thermostat. In the present discussion, an increase of the information 
entropy of the Smoluchowski process definitely occurs due to the thermal environment: the dif- 
ferential entropy is being generated (produced) in the physical system by its environment. 

Of particular interest is the case of constant information entropy 5 = which amounts to the 
existence of steady states. In the simplest case, when the diffusion current vanishes, we encounter 
the primitive realization of the state of equilibrium with an invariant density p. Then, b = u = 
DV In p and we readily arrive at the classic equilibrium identity for the Smoluchowski process: 

-{l/kBT)VV = V\n p (84) 

which determines the functional form of the invariant density in case of a given conservative force 
field, jHHllHHl- There is an ample discussion in Ref. [HE] of how these properties match with time 
reversal of the stationary diffusion process and the vanishing of the entropy production (in our 
lore, see e.g. [Hllinn]) rate (iS)j„. 

Coming back to the general discussion, let us define the so-called thermodynamic force Fth = 
v/D associated with the Smoluchowski diffusion and introduce its corresponding time-dependent 
potential function \I'(x, t): 

ksT Fth = F - kBTV In p = -V'i! . (85) 

Notice that v = — (l/m/3)V\l/. In the absence of external forces (free Brownian motion), we 
obviously get Fth = — Vlnp = —{1/D)u. 
The mean value of the potential 

<iJ = V + kBT\np (86) 

of the thermodynamic force associates with the diffusion process an obvious analogue of the 
Helmholtz free energy: 

{^) = {V)-TSg (87) 

where the dimensional version So = ksS of information entropy has been introduced (actually, 
it is a direct configuration-space analog of the Gibbs entropy). The expectation value of the 
mechanical force potential {V) plays here the role of (mean) internal energy, [7H I67j. 
By assuming that pVv vanishes at integration volume boundaries (or infinity), we easily get the 
time rate of Helmholtz free energy at a constant temperature T: 

^(^) = -kBTQ~TSG. (88) 
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By employing Eq. (j83|) . we readily arrive at 

(89) 

which either identically vanishes (equilibrium) or remains negative. 

Thus, Helmholtz free energy either remains constant in time or decreases as a function of time at 
the rate set by the information entropy "production" Sm- One may expect that actualy {'^)(t) 
drops down to a finite minimum as t — oo. 

However, this feature is a little bit deceiving. One should be aware that a finite minimum as well 
may not exist, which is the case e.g. for the free Brownian. Also, multiple minima need to be 
excluded as well. 



4.4 KuUback entropy versus Shannon entropy in the Smoluchowski process 



In the presence of external forces the property Eq. ()89|) may consistently quantify an asymptotic 
approach towards a minimum corresponding to an invariant (presumed to be unique) probability 
density of the process. Indeed, by invoking Eq. (|H^ we reahze that 

p4x) = lexp(-^) (90) 



where Z = J exp(— V"(x)/A;bT) dx, sets the minimum of (\l/)(t) at (\E')* = \E'* = — fc^TlnZ. 
Let us take the above p^{x) as a reference density with respect to which the divergence of p{x, t) is 
evaluated in the course of the pertinent Smoluchowski process. This divergence is well quantified 
by the conditional Kullback entropy Hc(t). Let us notice that 

He(t) = - j P^^ {j) dx = Sit) - In ^ - (91) 
Consequently, in view of Eqs. (|HH|) and (|H^. we get 

K = 5 + Q = (5),„ > (92) 

so that ^(^P) = —{kBT)'Hc- An approach of {'^){t) towards the minimum proceeds in the very 
same rate as this of Ti.c{t) towards its maximum. 

In contrast to He which is non-negative, we have no growth guarantee for the differential entropy 
S whose sign is unspecified. Nonetheless, the balance between the time rate of entropy produc- 
tion/removal and the power release into or out of the environment, is definitely correct. We have 
S > -Q. 

The relationship between two different forms of entropy, differential (Shannon) and conditional 
(Kullback), and their dynamics is thereby set. 

An exhaustive discussion of the temporal approach to equilibrium of the Gibbs and conditional 
entropies can be found in Refs. [n^l93j. Both invertible deterministic and non-invertible stochastic 
systems were addresses. 
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Typically, the conditional entropy either remains constant or monotonically increases to its maxi- 
mum at zero. This stays in conformity with expectations motivated by the Boltzamnn if-theorem 
and the second law of thermodynamics. 

To the contrary, the Gibbs entropy displays a different behavior, even under the very same 
approach-to-equilibrium circumstances: it may monotonically increase or decrease, and as well 
display an oscillatory pattern. In below we shall demonstrate that this potentially strange behav- 
ior of the differential entropy gives an insight into nontrivial power transfer processes in the mean, 
whose assessment would not be possible in terms of the conditional entropy. 

4.5 One-dimensional Ornstein-Uhlenbeck process 

It is quite illuminating to exemplify previous considerations by a detailed presentation of the 
standard one-dimensional Ornstein-Uhlenbeck process. We denote b{x) = —'jx with 7 > 0. 
If an initial density is chosen in the Gaussian form, with the mean value ao and variance Uq. the 
Fokker-Planck evolution Eq. ()74|) preserves the Gaussian form of p{x, t) while modifying the mean 
value a{t) = aoexp{—'jt) and variance according to 

a^(t) = exp(-27t) + — [1 - exp(-27t)] . (93) 

7 

Accordingly, since a unique invariant density has the form p^^ = ^/^yJ2T^D exp{—'yx'^ /2D) we 
obtain, P): 

2 

Hcit) = exp(-27t)K(po, P*) = exp(-27t) (94) 

while in view of our previous considerations, we have S(t) = (1/2) ln[27re(T^(t)] and JF = l/a'^(t). 
Therefore 

. ^ 27(/^-7^g)exp(-27t) 

D-{D- 7(t2) exp(-27t) ' ^ ' 

We observe that if (Tq > D/'-f, then S < 0, while ctq < -D/7 implies S > 0. In both cases the 
behavior of the differential entropy is monotonic, though its growth or decay do critically rely on 
the choice of ctq. Irrespective of ctq the asymptotic value of S(t) as t 00 reads (1/2) ln[27re(D/7). 
The differential entropy evolution is anti-correlated with this of the localization, since 

= — f96) 

For all (Jq the asymptotic value of J-' reads 7/-D. 

We have here a direct control of the behavior of the "power release" expression Q = Hc — S. Since 

= {i^al/D) exp(-27t) > , (97) 

in case of 5 < we encounter a continual power supply Q > by the thermal environment. 
In case of 5 > the situation is more complicated. For example, if = 0, we can easily check 
that Q < 0, i.e. we have the power drainage from the environment for all t G R^. More generally, 
the sign of Q is negative for < 2{D — 7(To)/7. If the latter inequality is reversed, the sign of Q 
is not uniquely specified and suffers a change at a suitable time instant tchange{cio, c"o)- 
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4.6 Mean energy and the dynamics of Fisher information 

By considering {—p){x,t) and s{x,t), such that v = Vs, as canonically conjugate fields, we 
can invoke the variational calculus. Namely, one may derive the continuity (and thus Fokker- 
Planck) equation together with the Hamilton- Jacobi type equation (whose gradient implies the 
hydrodynamical conservation law Eq. ()76|) ): 

dtS + ^{Vsy-{n-Q)=0, (98) 

by means of the extremal (least, with fixed end-point variations) action principle involving the 
(mean Lagrangian: 



C = - p 



1 fu^ 



dx . (99) 



The related Hamiltonian (which is the mean energy of the diffusion process per unit of mass) 
reads 



H= p- 



dx (100) 



1. e. 



n = {i/2){{v')-{u'))-{n) . 

We can evaluate an expectation value of Eq. (jnH|) which implies an identity 7i = — {dts). By 
invoking Eq. ()86|). with the time- independent V, we arrive at 

^ = -^V(vp) (101) 
P 

whose expectation value in view of f p = at the integration volume boundaries, identically 
vanishes. Since v = — (l/m/9) V\l/, we define 

s{x,t) = (l/m/?)^(x,t) =^ {dts) = (102) 

so that H = identically. 

We have thus arrived at the following interplay between the mean energy and the information 
entropy "production" rate: 

generally valid for Smoluchowski processes with non-vanishing diffusion currents. 

By recalling the notion of the Fisher information Eq. P7jl and setting = D^Ta-, we can rewrite 

the above formula as follows: 

= {v^) -2{^) >Q (104) 

where jF/2 = — {Q) > holds true for probability densities with finite mean and variance. 
We may evaluate directly the uncertainty dynamics of the Smoluchowski process, by recalling that 
the Fisher information !F/2 is the localization measure, which for probability densities with finite 
mean value and variance cr^ is bounded from below by l/cr^, see e.g. Section III. 
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Namely, by exploiting the hydrodynamical conservation laws Eq. (|76|) for the Smoluchowski process 
we get: 

dtifw^) = -V ■ [{pv^)] - 2pv ■ V(Q - n) . (105) 

We assume to have secured conditions allowing to take a derivative under an indefinite integral, 
and take for granted that of pv^ vanishes at the integration volume boundaries. This implies the 
following expression for the time derivative of (f^): 

l{v') = 2{vVin-Q)) . (106) 

Proceeding in the same vein, in view of Cl, we find that 

|(^]) = (^;■Vfi) (107) 
and so the equation of motion for JF follows: 

jT=j^[{v')-2{Q)] = -2{vVQ). (108) 

Since we have VQ = VP/ p where P = D^p A In p, the previous equation takes the form = 
— / pvVQdx = — J vVPdx, which is an analog of the familiar expression for the power release 
{dE/dt = F ■ V, with F = —W) in classical mechanics; this to be compared with our previous 
discussion of the "heat dissipation" term Eq. (|H^. 

Remark 11: As indicates our previous example of the Ornstein-Uhlenbeck process in one dimen- 
sion, there is nothing obvious to say about the growth or decay of various quantities involved. In 
this particular case, we have e.g. (f^)(t) = {D /2)'Hc = t{'y'^al/D) exp(— 27t), hence an asymptotic 
value 0, while (M^)(t) = {D/2)J^{t) -f/D. Accordingly, we have (^])(^) --f/2D. 



5 Differential entropy dynamics in quantum theory 

5.1 Balance equations 

In the discussion of Smoluchowski diffusions, our major reference point was the conventional 
Fokker-Planck equation (f7^ for a probability density supporting a Markovian diffusion process. 
The (time-independent) drift function b was assumed to be known a priori (e.g. the conservative 
external forces were established on phenomenological or model construction grounds), while the 
initial and/or boundary data for the probability density of the process could be chosen (to a high 
degree) arbitrarily. 

Under such "normal" circumstances, the hydrodynamical conservation laws ()7fi|l come out as a 
direct consequence of the Fokker-Planck equation. Also, the functional expression for Q of the 
form ()77|1 is basically known to arise if one attempts to replace an elliptic diffusion operator by a 
Hermitian (and possibly self-adjoint) one, jHHl EH ESI • 

We shall depart from the standard Brownian motion setting to more general Markovian diffusion- 
type processes which, while still respecting the Fokker-Planck equation, admit general time- 
dependent forward drifts. In fact, we invoke at this point a well defined stochastic counterpart 
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of the Schrodinger picture quantum dynamics of wave packets, jHDl IHH IHHl IHll 1121 IHHl EHj, where 
the notion of differential entropy and its dynamics finds a proper place. The dynamics of quantal 
probability densities is here resolved in terms of diffusion-type processes. 

Let us assume to have chosen an arbitrary continuous (it is useful, if bounded from below) function 
V = V("af,t) with dimensions of energy, we consider the Schrodinger equation (set D = h/2m) in 
the form 

idt^J = -DAij + • (109) 

The Madelung decomposition ip = p^/^ exp(is) with the phase function s = s{x, t) defining v = Vs 
is known to imply two coupled equations: the standard continuity equation dtp = — V(fp) and 
the Hamilton- Jacobi-type equation 

dtS + ^iVsf + in-Q) = (110) 

where Q = V/m and the functional form of Q coincides with this introduced previously in Eq. (|7^ . 
Notice a "minor" sign change in Eq. piO|) in comparison with Eq. (j^Hj) . 

Those two equations form a coupled system, whose solutions describe a Markovian diffusion-type 
process: the probability density is propagated by a Fokker-Planck dynamics of the form Eq. ((73]) 
with the drift b = v — u where u = DV In p is an osmotic velocity field. 

We can mimic the calculus of variations steps of the previous section, so arriving at the Hamiltonian 
(actually, the mean energy of the quantum motion per unit of mass): 



1 /n' 



dx, (111) 



to be compared with Eq. (jlOOj) . There holds 

n = {i/2)[{v') + {u')] + {n) = - (dts) . (112) 

Of particular interest (due to its relative simplicity) is the case of time-independent V, when 

n = - {dts) = £ = const (113) 

is known to be a conserved finite quantity, which is not necessarily positive. Since generally Ti ^ 0, 
we deal here with so-called finite energy diffusion- type processes, jHH E2]- The corresponding 
Fokker-Planck equation propagates a probability density \ip\^ = p, whose differential entropy S 
may quite nontrivially evolve in time. 

Keeping intact the previous derivation procedures for {S)in (while assuming the validity of math- 
ematical restrictions upon the behavior of integrands), we encounter the information entropy bal- 
ance equations in their general form disclosed in Eqs. (jST|) - ()83p . The related differential entropy 
"production" rate reads: 



{<S)in — 



>0,. (114) 
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We recall that |jF = —{Q) > which implies S — {Q) > |jF > 0. Therefore, the localization 
measure has a definite upper bound: the pertinent wave packet cannot be localized too sharply. 
We notice that the localization (Fisher) measure 

= 2{S - (Q)) - (v^) (115) 

in general evolves in time. Here £^ is a constant and 17 = 0. 

By invoking the hydrodynamical conservation laws, we find out that the dynamics of Fisher 
information follows an equation: 

^ = +2{vVQ) (116) 

and that there holds 

which is to be compared (notice the opposite sign of the right-hand expression) with the result we 
have obtained for Smoluchowski processes. 

Obviously, now we have JF = + J vVPdx, with the same functional form for P as before. We 
interpret as the measure of power transfer in the course of which the (de)localization "feeds" the 
diffusion current and in reverse. Here, we encounter a negative feedback between the localization 
and the proper energy of motion which keeps intact an overall mean energy Ti = £ oi the quantum 
motion. See e.g. also |53] . 

In case of w = 0, we have £ = \T \ {Q) and no entropy "production" nor dynamics of uncertainty. 
There holds S = and we deal with time-reversible stationary diffusion processes and their 
invariant probability densities p{x), 15 2 j . 

Remark 12: Let us indicate that the phase function s{x,t) shows up certain (remnant) features 
of the Helmholtz \Ef and (\E'). This behavior is not unexpected, since e.g. the ground state densities 
(and other invariant densities of stationary states) are directly related to time-reversible stationary 
diffusion-type processes of Refs. [521 EHl- We have —{dts) = £. In view of f = Vs and assumed 
vanishing of spv at the integration volume boundaries, we get: 

±{s) = {v')-£ (118) 

The previously mentioned case of no entropy "production" refers to f = and thus s = Sq — £ ■ t. 
We recall that the corresponding derivation of Eq. (jHn|) has been carried out for v = — (l/m/3)V\E', 
with = 0). Hence, as close as possible link with the present discussion is obtained if we 
re-define s into s^, = —s. Then we have 

f^{s^) = £ - {v') . (119) 

For stationary quantum states, when v = identically, we get ^(s*) = £, in contrast to the 
standard Fokker- Planck case of ■^{'^) = 0. 

Interestingly enough, we can write the generalized Hamilton- Jacobi equation, while specified to 
the V = regime, with respect to s.^. Indeed, there holds 9ts^ = Q — Q, in close affinity with 
Eq. (jUHj) in the same regime. 
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5.2 Differential entropy dynamics exemplified 
5.2.1 Free evolution 

Let us consider the probability density in one space dimension: 

a ( x^o? 



and the phase function 



, , 2D^xH . f 2Dt\ 
Ka^'^) = ^^7^-^ arctan [121] 



which determine a free wave packet solution of equations p()9|l and piop . i.e. obtained for the 
case of V = with the initial data ip{x,0) = (vra^)"^/^ exp(— x^/2q;^). 
We have: 

w N .X . s 2D(2Dt-a^)x , , 

t) = vix, t) + t) = ^,^^j^2t2 (122) 

and the the Fokker-Planck equation with the forward drift b{x, t) is solved by the above p. 
In the present case, the differential entropy reads: 

5(t) = ^ln[2vre(X^>(t)] (123) 

where (X^) = / x'^pdx = (a^ + 4DH'^)/2a'^. Its time rate DS = {v"^) — {b ■ v) equals: 

dS ADH , , 

for if: > 0. Its large time asymptotic is D jt. 
Furthermore, we have 

with the obvious large time asymptotic value 2D'^ /a^: the differential entropy production remains 
untamed for all times. 

Due to (^2) = {2D^a^)/{a^ + ADH'^) there holds 

^ = \{{v') + {y')) = ^- (126) 

Accordingly, the quantum mechanical analog of the entropy (rather than heat) "dissipation" term 
—D- Q in the quantum case reads 

„ , ADHia^-2Dt) , , 

and while taking negative values for t < a^/2D, it turns out to be positive for larger times. 
Formally speaking, after a short entropy "dissipation" period we pass to the entropy "absorption" 
regime which in view of its D/t asymptotic, for large times definitely dominates D{S)in ~ 2D'^ jo? . 
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Those differential entropy balance features do parallel a continual growth of the mean kinetic 
energy (1/2) (t;^) from an initial value towards its asymptotic value D'^/a^ = S. Note that the 
negative feedback is here displayed by the the behavior of (m^) which drops down from the initial 
value 2D^/a^ towards 0. It is also instructive to notice that in the present case J-'{t) = D"^/ (X^)(t). 
We can readily check that = d{v?)/dt = —d{v'^)/dt. 

5.2.2 Steady state 

We choose the probability density in the form: 

UJ \l/2 



^^^'^^ = l2^J exp (X - g(t)) 



:i28i 



where the classical harmonic dynamics with particle mass m and frequency uo is involved such 

that q{t) = go cos{ujt) + {po/muj) sin(a;t) and p{t) = po cos{ujt) — muqo sin(to't). 

One can easily verify that (9), and (40) hold true identically once we set V = ^cu^x^ and consider: 

s{x,t) = (l/2m) [xp{t) - {1 /2)p{t)q{t) - mDut] . (129) 
A forward drift takes the form: 

b(x,t) = —p(t) -uj(x -q(t)) (130) 
m 

and the above p solves the corresponding Fokker-Planck equation. 

The differential entropy is a constant equal S = (1/2) ln(27reD/co'). Although trivially dS/dt = 0, 
all previous arguments can be verified. 

For example, we have v = Vs = p{t)/2m and therefore an oscillating entropy "production" term 
D{S)in = p^(t)/4m^ which is balanced by an oscillating "dissipative" counter-term to yield S. 
Suitable expressions for (s) and (dts) easily follow. 

Concerning the Fisher measure, we have obviously J-' = u/D which is a constant of motion. 
5.2.3 Squeezed state 



Let us consider j33] the squeezed wave function of the harmonic oscillator. We adopt the re-scaled 
units h = uj = m = 1, hence also D = 1. The solution of the Schrod inger equation idtip = 
-{1/2) Alp + {x^/2)tp with the initial data ip{x,0) = (7^)"^/^ exp(— x^/27^) and 7 G (0, cxo), is 
defined in terms of the probability density : 

^^^'^^= (2vr)VMt) '"P (~^) ^'''^ 



where 



and the phase function 



2(T^(t) = i-sint + 7^cos^t (132) 
T 



t 71 . 9 X (1/7^ — 7^) sin2t r, 
s{x, t) = - + - + arctan(72 cot t) + ' i . (133) 
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Now, the differential entropy S = (1/2) ln[27recr^(t)] displays a periodic behavior in time, whose 
level of complexity depends on the particular value of the squeezing parameter 7. The previously 
mentioned negative feedback is here manifested through (counter)oscillations of the localization, 
this in conformity with the dynamics of cr^(t) and the corresponding oscillating dynamics of the 
Fisher measure ^ = l/cr^(t). . 

See e.g. also jHSl for a pictorial analysis and an instructive computer assisted discussion of the 
Schrodinger cat state (superposition of the harmonic oscillator coherent states with the same 
amplitude but with opposite phases), with the time evolution of the corresponding differential 
entropy. 



5.2.4 Stationary states 

In contrast to generic applications of the standard Fokker-Planck equation, where one takes for 
granted that there is a unique positive stationary probabihty density, the situation looks otherwise 
if we admit the Schrodinger equation as a primary dynamical rule for the evolution of (inferred) 
probability densities. For a chosen potential, all available stationary quantum states may serve the 
purpose, since then we have nonnegative (zeroes are now admitted) p*(a;), and v{x) = identically 
(we stay in one spatial dimension). 

The standard harmonic oscillator may serve as an instructive example. One may e.g. consult Fig. 
3 in pU] to check the behavior of both position and momentum differential entropies, and their 
sum, depending on the energy eigenvalue. All these stationary state values grow monotonically 
with n = 1,2,..., 60, [SO] and follow the pattern in the asymptotic regime n = 500, [HHI- 
For convenience we shall refer to the Schrodinger eigenvalue problem with scaled away physical 
units. We consider ( compare e.g. Eq. ()109j) with D 1/2) 

[-^A + ^]^, = in + ^)Vp:. (134) 

In terms of a suitable Hamilton- Jacobi type equation we can address the same problem by seeking 
solutions of an equation 

n + l/2 = Q-Q (135) 

with respect to ^/p^, provided we set Q = define u = Vln^/p7 and demand that Q = 

+ (1/2)V ■ u. 

For the harmonic oscillator problem, we can refer to standard textbooks. For each value of n we 

1 /2 

recover a corresponding unique stationary density: ^/p7 ^ Pn with n = 0, 1, 2, ...). We have: 

''""<^' = (2-n!^)V. (4) *"<^' ^'''^ 

where Hn{x) stands for the n-th Hermite polynomial: Hq = 1, Hi = 2x, H2 = 2{2x'^ ~ 1); 
H3 = 4:x(2x'^ — 3), and so on. 

We immediately infer e.g. feo = ~^ ^ Q = x^/2 — 1/2, next bi = {1/x) — x Q = x^/2 — 3/2, 
and 62 = [4:x/{2x^ - 1)] - x ^ Q = x^/2 - 5/2, plus 63 = [{l/x) + 4:x/{2x^ -3)]^Q = x'^- 7/2, 
that is to be continued for n > 3. Therefore Eq. ()135|) is here a trivial identity. 
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Obviously, except for the ground state which is strictly positive, all remaining stationary states 
are nonnegative. 

An open problem, first generally addressed in [^3], see also I^S], is to implement a continuous 
dynamical process for which any of induced stationary densities may serve as an invariant asymp- 
totic one. An obvious (Ornstein-Uhlenbeck) solution is known for the ground state density. Its 
ample discussion (albeit without mentioning the quantum connotation) has been given in Section 
IV. 

6 Outlook 

One may raise an issue of what are the entropy functionals good for. In non-equilibrium statistical 
mechanics of gases one invokes them to solve concrete physical problems: for example to address 
the second law of thermodynamics and the related Boltzmann if-theorem from a probabilistic 
point of view. 

We are strongly motivated by a number of reported problems with the rigorous formulation of 
effects of noise on entropy evolution and a justification of the entropy growth paradigm for model 
systems, jSHl IHZl tHHl 1121 UHl ini] and a long list of mathematically oriented papers on the large time 
asymptotic of stochastic systems, |US]- |102j . Therefore, the major issue addressed in the present 
paper is that of quantifying the dynamics of probability densities in terms of proper entropy func- 
tionals. The formalism was designed to encompass standard diffusion processes of non-equilibrium 
statistical physics and the Schrodinger picture implemented dynamics of probability densities re- 
lated to pure quantum states in L'^{R). In the latter case an approach towards equilibrium has 
not been expected to occur at all. 

To this end the behavior of Shannon and Kullback-Leibler entropies in time has been investigated 
in classical and quantum mechanical contexts. The utility of a particular form of entropy for 
a given dynamical model appears to be basically purpose dependent. The use of the Kullback- 
Leibler entropy encounters limitations which are not shared by the differential entropy, when the 
dynamical process is rapid. Alternatively, if one is interested in its short-time features. 
On the contrary, it is the conditional Kullback-Leibler entropy which is often invoked in rigorous 
formulations of the Boltzmann if-theorem under almost-equilibrium conditions and its analogues 
for stochastic systems. The large time asymptotics of solutions of Fokker-Planck equations, if 
analyzed in terms of this entropy, gives reliable results. 

However, our analysis of Smoluchowski diffusion processes and of the exemplary Ornstein-Uhlenbeck 
process, demonstrates that a deeper insight into the underlying non-equilibrium physical phenom- 
ena (the inherent power transfer processes) is available only in terms of the Shannon entropy and 
its time rate of change. This insight is inaccessible in terms of the Kullback-Leibler entropy . 
The differential entropy needs not to increase, even in case of plainly irreversible dynamics. The 
monotonic growth in time of the conditional Kullback entropy (when applicable), not necessarily 
should be related to the " dynamical origins of the increasing entropy" , ^3 EB] • We would rather 
tell that the conditonal entropy is well suited to stay in correspondence with the lore of the second 
law of thermodynamics, since by construction its time behavior is monotonic, if one quantifies an 
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asymptotic approach towards a stationary density. 

In case of Smoluchowski processes, the time rate of the conditional KuUback entropy was found to 
coincide with the corresponding differential (Shannon) entropy "production" rate. The differential 
entropy itself needs not to grow and may as well change its dynamical regime from growth to decay 
and in reverse, even with the entropy "production" involved. 

Balance equations for the differential entropy and the Fisher information measure involve a non- 
trivial power transfer. In case of Smoluchowski processes this power release can be easily attributed 
to the entropy removal from the system or the entropy absorption from (drainage) the thermostat. 
In the quantum mechanical regime, the inherent power transfer is related to metamorphoses of 
various forms of mean energy among themselves and needs not the notion of external to the system 
thermostat. 

Apart from the above observations, we have provided a comprehensive review of varied appear- 
ances of the differential entropy in the existing literature on both classical and quantum dynamical 
systems. As a byproduct of the general discussion, we have described its specific quantum manifes- 
tations, in the specific (pure quantum states) regime where the traditional von Neumann entropy 
is of no much use. 
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